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AUTOMATED TEST ITEM BANKING 



h INTRODUCTION " 

The high speed digital computer has proven to be an important labor saving device for the 
psychometricijin. For two decades, tests and items havq been analyzed with speed and accufacy, and the 
number and quality of item analysis programs have increased every year. With the relative cost of computer 
computations decreasing, automated item analysis has become the norm. Recendy, adaptive aptitude ind 
achievement tests have been administered ort the computer (Bejar, Weiss, & Giallxica, 1977; Ree,- 1977). 
These adaptive testing procedures have required^ computer maqipulabljMtem pools and sophisticated- 
programming algorithms to facilitate item storage ^d retrieval. - - 

An outgrowth of computer adaptive testing and item analysis is the computer storage of test items- 
and ^il^eractive test construction. Lippey 11974) presents 'several alternative concepts of this **computer 
assisted lest CO nstructiorj^^* ^ . 

Test cofl*truction is a time-consuming ari(J laborious task whicb requires the methodological searching 
lof files of item cards and the selection of relatively few items:from a large mass of Items. During this search 
the iteins must be sorted and resorted and, as the number of items increases, so-does the probability of not 
selecting the best available item. The selected.items must be ordered by difficulty or in some other fashion 
and then edifed for final form. Next, this final group of items must be typec^and reproduced into booklets 
for experimental pi^rposes. The Experimental booklets must then b6 administered and scored. Finally, the 
test statistics must be computed arid the appropriate norms deivelopec}. - • 

During the test'constru.ctiOn phase, the security of the t^s\ items, answer keys,' and the test must be^ 
/maintained-. The method described in this report permit^ the interactive construction of tests and. 
manipulation of banks of test items at a computer terminal. Itis a' system which pemiits maximum iecUfity" 
by using computer files available ^only to specific users. The system permits a test constructor to search 
through a large number of items very rapidly and without the clerical errors frequently involved injmanual 
searches. Test items may be printed several different -ways. The system ^so provides the user with a stop 
.and restart procedure and keeps track of items selected, their relevant characteristics, and how many more 
items are required to build the test. All thpse capabilities permit more rapidi secure, and/error-free test 
construction. • ^ ^ 

' IL THE SYSTEM ' 

'4 ' , ^ ■ 

" ' The computer can assist test preparation in the following four ways: (a) stor^or banking of test 
items, (b) banking of item attributes, (c) test construction,^ and (d) test printing. Each of these capabilities 
is requ^e^ for test construction. . 

XheVutomated Item Banking (AIB) system is comprised of three modules: (a) a text editor which 
permits the\stablishment and/pr modification of item banks, (b) an item bank listing proced<lre which 
permits' selected items from the item bank to be printed on the high-speed printer, and (c) an intcj live 
test construction procedure which permits thfe interactive searching of an item pool, the construction^f a 
test, and the estimation of test statistics. 

The text editing processor is designed with a series of options which permit-'the interactive user to 
establish an item pool, modify items, change item statistics, modify or replace item stems or item-options, 
ai^ obtain printed copies of specified iteips. It is useful for. updating item banks and for adding items to 
existing item banks. Th6 files^ created by the text editm are designed to be parsimonious of computer 
storage space and are easily modifiable by appropriate use?s but are secure againsf unauthorized tamperi 



The listing procedure is' used to provide printed copies of either aB or a specified subset of the items 
and item statistics int an Jtem pool.^Printed copies of the items from the jisting* procedure ^re useful for 
purposes of proofreading, error detection, and transfer of the items to another facility or organization. 

^ The test construction routine perrnits tlie interactive searching of an item poof ai;id the selection of 
items'. Operational or experimenj^ tests may be built "on line'' by specifying desired itenTstatistics. A 
search based on either classical item difficulty (p) or latent-t^it difficulty (b) attributes may be specified. 
/As the'f?xt 0^ an item is interactively presented on the compCit^r terminal', the item statistics, keye'd 
response^ normative sample, and last operational use of the item are also presented. The item is then either 
accepted foronclusion in the test or rejected and another item found. • 

During test cortstnJction,^the routine estimates test statistics for the subset of items selected and 
keeps track of the numb^ of items initially requested and. the numbeu; of items remaining tCKbe selected.- 
An option for suspending aW ijps'tarting'the.test construction^procedureis also provided. 

Finally, the printing of the interactively. constructed test is done ori^the highspeed line printer in one 
of three optional way^: the jitems ^riiay be printed in the order selected, sorted according to classical 
difficulty, or sorted according to latent-trait difficulty. Se^ Appendik A for a detailed explanation of .the^ 
theory of the operation of the system, and Appendix B for computer documentation." , 

* The system's computer;,^ff6grams are written ip FORTpilAN and assembly language^ All^file handling is 

done in assembly level language to reduce overall/program , size. It is iniplemented on- a Univac 1108 with 

level 32R2B of the Exec 8 executive system. . - ' ^ . >^ 

' * V . • * ^ 

, ' . ' '.^ ■ - , . ^ - • 
— V in. THE re;search4^roblem ^ . _ 

In order foj^such an auton^ted system to be oi^any value, it must be accurate, as well as fa^The 
following study was conducted in order to determine me accuracy of the estimation procedures. Of interest 
in this study were"the comparisons between^ the ^stmiated and actual values for the test mfean, standard 
devi^^on, and reliattUity of an interactively constructed test; * 

■ " ■ ^' ' ' ' ■■■■ " 

IV. METrtOD 

Subjects ' * ^ ^ 

A group of 1,502 basic airman trainees at Lackland AFB, Texas, was-rando^ly selected for^this 
study. The s^ibjects were randomly assignee^ to either the developmental group oi the cross-validation 
group. * ^ r ^\ ^ ^ 



Items , [ > 

^ ■■ ^ '\ ^ , 

The items were the Word Knowledge type cuHed from existing item files and were all of the »same. 
form. Each item consisted of a stem containing a word and five item options. The examinee was asked to 
select the item-option which most nearly means the same as the stem word. An example is presented as 
follows: • - 

"Small means 

' 1.. sturdy. 

2. round. 

3. cheap. , 

4. little. 

5. crisp." ^ 
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, Items- of this type are frequently found in milit^jy selection and classification test batleries as well^ 
iji.acad<emic ap^tude and vocational aptitude tests. , . 

Using ^aV^ilable item difficulty (p) and item-test phi coefficients^an experimental test was designed^ 
cover the difficulty (g) range of about .20 to about :9a with ;He higliest available phi coefficients in each 
difficulty level. An exparimental test' of 200 of these items was assembled andadministered to the subjects 
in a counterbalanced manner witfi half the subjects beginning with item 1 and half beginning with item I'Ol . 
The subjects were admmistered all 200 items in one continuous time period. * 

• 1 ■ . 'A . O 

Item Calibration ' 1 

^ The it^' responses from the^ devotopmental group weje^used to estimate the latent-trait item* 
parameters (Urry, 1975), the classical^item difficulty (p), and item discriminatidn (item-test biserial 
^correlation) indices. From the group of 200 items, sever?[l were discarded due to' poor item characteristics 
or due to defensible distract or^. A final pool of 1 IS^tems was entefed into the AIB system. 

■ ^ , ■ ■ 1, 

Automated Item Bank Trvout - 

r . • c\ 

A test outline was developed to emulate the structure of a proposed test. Tlie AIB system was used to 
search the it*i bank for item difficulty (p) Values between .20 and .90. Table 1 shows the distribution of 
desired p values. , * . , V 

■ -t ' • * " 

• '^ ^ ^ - Table 1. Distribution of Item Difficulty 

Values Requested of AIB for the , 
* SI ' Experimental Test . ' 

.■ • J • — — 



»ltem Difficulty , ■ ■ Item!! Requested \ ^ 



.20 - .29 


' 1 


.30 -.39 • 


2 


.40 - .49 


4 


.50 - .59 


6 


.60 - .69 


4 


.70 - .79 \ 


. . 2 


.80 - .89 ^ 


1 


^ Total 


20 



The AIB system was Activated, a n^ue search was initiated, and 20 items were select^. Using the 
printing option, the items and the esttaated 'test statistics' were output to the high-speed pilfer. These^ 
selected items w^re then identified in tW 20a-jtem experimental* Word Knowledge test, antf these •20 items'^ 
were scored for the hold-out or cross-valiaity group. ' ^ ^. " ' t** - 



V. RESULTS 



' mie 2 presents Ji comparison of the hold-out group calcul3ted and AIB estimated characteristics for 
the 20-item AIB developed test. - / ' 



Table 2. Calculated and Estimated Test Values 
for 20 AIB Selected Items 



Qfelculated v. 

' (Based on ' 

HoldH>ut Group AIB ^ 

Test Statistics lu Ns7%1) Ertlmated 

MeanS(:.ore • ^ 10.48 10.62' 

Standard Deviation ' 4.26 , ' ^ \ f 00 

ReliabUity^ .80 . ' " ' . -.72 

A Coefficient Alpha measure of internal consistency. ^ * \ ' 



Vl. DISCUSSION. 



Comparisons of the estimated versus the computed statistics indicate that the procedures are accurate 

and permit the test constructor to build a'test with desired psychometric characteristics. 

* ■ . ^> * ^^^^ 

. All 20 items fleeted interactively were sele9ted by an individual with^ no computer programming 
experience. The e^ase' of use of the AIB system is in large part due to the use of patural language commands. 
In addition/'the formatting or positioning of these commands on the computer'temiinal is inconsequential. 
The computer asks a question in plain English, a5nd thej^Jpfeer* responds in plain English. User errors in 
responding are handled by displaying on the computer terminal the inappropriate command response and a 
list- of the appropriate comman'ds. . ' - . . 

The AIB system operates rapidly and permits a high degree of user flexibility. 

^ yil. CONCLUSIONS - 

The AIB system can be utilized to construct tests of known quality "on line." Use of this system' can 
reduce storage of item cards, and proper storage o/ item banks orvmagnetic. data tapes and/or microfiche 
can provide protection against loss. Input opefation and the automated selection proced^ires are simple 
enough to be used by those without sophisticatiorf in computer programming or operation. The speed and 
flexibility of the^ AIB system' permit the construction of tests in le^time and with* reduced security 
problems.' ' , ' " ' " . . ^ " ' 
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APPENDIX A: TECH>4ICAL ASPECTS OF THE AUTQMATED^ITEM BANK ' 
> ^ \ / ' ' 

The automated item, bank iJermits tv^o types of searches biit always estimates Jhe test statistics based 
in latent-trait theory. When a, search is initiated, .a b or p value is requested and an interval is created around ^ 
that value. For exaniple, if a p value of .50 is (Requested, an interval from .47 to .53 is created; then the 
vector of item p values is searched and tfte first item with-a p value in this interval is displayed. If no' item 
with a p value in this interval is found, then the interv^ is increased by a coi}"§tant .03. TTien the interval is 
'searched again until an iterp is located or the'^interval is widened again and searched until an'iferh is found, , 
or until the p value interval is from .00 to 1 .00^ and thp system issues an error message an^shu^s down. The 
same 'general procedure is followed for a b value seardi except that the constant is ..10 and th'e termination 
bounds are set at —3.1 and -^3.1. ^ [/ ^ ^" " 

^ When an item is presented for consideration by^the^testA:onstructor, it m*ay be accepted or rejected 
by answering the termin4 prompt question with- a natural language ^*¥fig*2__Qj "NO." An affirmative 
response includes the item in the tentative te^^d a negative response sets the program to searching the 

item bank file again. ^^O'^""'^^ ' , ■ ^ 

i> '\ . . " h , ' , 

After^The second iipm has beep included in the test, a routine estimates the expected values for the ' 
test mean, variance, staFWafd .deviatiory and test reliabyiiy for two groups of **pseudo subjects." Group 1 is 
defined as a full range sample ^oft.he mobilization«pQpulation and covers the^ fjrst to the 99th percentile. 
'Hie second group is a restricted sample which runs frorh the 30th percentile to the 99th percentile of the. . 
same population. The SOthe percentile was selected because it^s^bout the lowest value encountered" in Air 
Force enlisted recruits. Jt xemld^ set to any other value in the progr^ to rejject a different lower limit. 
.This estimation procedure is acppmRlished by generating 5^000„unit normal ratifloni^bility estimates," 0, 
in TRe interval of about -3.0 to *H-3.0 and then using equation (l).generatin^he probability that a ' 
**subject" with that Q would pass the item. This is done for each "subject/.' across all the items in the test. 

- P(0) = c+(l-'c) + (lVe(-l-7a(9-l^)))~' ' . V: ' \ . 

where P(^) is the probability of passing the item, a is th^item discrimination parameter, t>is.the item 
difficulty or location parameter, and ^c is the lower asymptote of the item characteristic curve>See 
Birnbaum (1968) Tor a compHefte discussion of the 3 parameter logistic latent-trait model. The variance of 
subjecf scores -is computed ajid also the ^it em variance. From^ this informati6n a Coefficient >^ha 
(Cronback, 1951) reliability estimate 'is computed. This procedure is repeated after each itefrr is seliected 
afad when th# test construction phase is completed. An estimate of expected percjentile equivalent ^"^s is 
computed usii^ equation (2). - . ■ ^ 

Percentile = 0(Z) + K ■ ' V ... ^ ^ ^ (2) 



where Q is the estimated, test standard deviation, K is tbe estimated mean, and Z is the unit normal Z 
score cor^sponding to specific percentiles in the normal distribution. ^ • . 

The next pjpcess of the^'AIB is to updSe the item fdes with the nuinber of the operational test 
constructed for those items used in the test (see Appendix B). ^ ^ ^ ^ 

The last procedure of the AIB is to offer the test constructor a choice of output ordering of the 
selected test items, the actual test items, the final test statistics estimates, and the percentile equivalents. 
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■ APPENDIX B: OPERATION OF THE AUTOMATHD ITEM B A NIC 



Uem iJaiiking Operations ' r y . • 

Iliis autoniated bankjng.and test construction sVstcni is-desi^d forahc UNIVAC IIjOO series. It 
is currently implemented on the UNIVAC 1108 with EXEG 8 aTlevcl 32R2B. 

The system inus.t be used with previously 'catalogued item fdes. and each scparaj^rocedure is 
executed in tjie same iiKumer with a processor call foliowed by t lie r ure of ilv i^, " 

-'FILE. ELEMENT ITEMFILE. l^- 

Input errort> cause the input prompt to be. repeated along with an ecoti Hiossai'.c. 

'rerminatipn of a run dways causes an e.\it statement to be printed so that the operator kiuuvs tfie 
system has been successfully exited. . ' , 

Any termination of a run prior to tiie con'ipletion of tiie requested ;Vjtion causes a fife to^be 
autamatically cataloRUcd for a restart at -/Mue future date. ■ " " ' " 

Each iteni nlr -.h.-ijld contain otdy one item type such a-; WORD KNOWLEIXih;. Do not nrix 

ability areas. ' -. 

Abbreviations used in, item bank files. 

K = number of^fie correct item option 
P ~ portion of examinees selectin[^ keyed option 
B, Eitent-trait (liffieuity or location parameter - 
J)' median-split item test [)iii eoetlicicnt • . ' 

RBI - itciu-test I>iserial correlat'<Mi ' ' sJ - 

A =. latent-trait discrimination pLirameter 
JIFB-^ item-test point biserial correlation 

"C latent-trait '*gi;\>ssing" parameter • . - 

SMPL --^ identincation of item-analysis saiople (see para 4 for number code) 

TEST = Identificaticm (d~ operational test m vAw^U item Ljsf a^^pearcd (see [lara for mimber 
co'de). 



SMFL -.identification nmnber;. I-irst number in'dicab;s oivivtional test type. Dc.cimal [H)rti()n of 
nund:>ei indicates wliicii form of (operational test .■ ' < 

identification of Arnkul Services VovMiiona.l Aiititmic |->atr/iy ( ASVAB). 

'l.OO ASVAB>ypc items ' ^ , / ' 

1.01 - used in ASVAB One 

1.02 used in ASVAii Two 

FO.V used in ASVAB Three ^ 
1.04 • used in ASVAB boiii 
r.05 used in ASVAB I4ve 
i.U() useil in ASVAB SJix^ 
1.07 useil in ASVAB Seven 

He. ^ . S • ' : ^ 

hienlitlcation.of I-nlistment Scieenin|'/rest (I'ST) 

2.00 F,$i:,jg^|K items 

2.01 liST Form One^ . - ' 

2.02 i:S I' l'(Hm Fw,,* . jt ' " , ' 



identification of Electronic Data Processing^est (EDI*T) 

3.00 -"EDPT type items 

3.01 - EDPt Fprm One . 

3.02 - EDPT#orm Two " . ' • 



Identification of Air Force Offict^r Qualifying Test (AFOQT) 

5.00 - A'^QT type items 

5.01 - AFOQT Form M 

5.02 ~ A|OQT Form N 

5.03 ~ aI^T Form 0 

List of Acceptable Numbers for SMPL J 

Identification of ASVAB Samples 

1.01 — unselected basic airmen 

1.02 - unselected male basic ainiVcn 

1.03 -.unselected female basic airmen 

1.04 — unselected. Wliite 'basic airmeti 

1.05 " unselected npn-White basic airmen 

Idcnti/icalioii of FST Samples^ 

2.01 - unsclcctcd basic airmen 

2.02 - unsclcctcd male basic airmen 

2.03 - unsclcctcd fem'aic basic airmen 

2.04 ~ unselected Wliite basic airmen 

2.05 - unselected non -White basiCj^ainiien ,. , 

Identification of FDPT Samples"" 

3.01 unsclcctcd basic airmen' 

3.02 unsclcctcd male basic airmen V 

3.03 - unsclcctiji^cm;dc basic airmen 

3.04 - unscloct^Wlii'te basic airmen 

3.05 unsclcctcd non-White basic airmen ^ 

ft . • . 

Identification ol" iWT Samples 

4.01 • unsclcctcd basic airmen 
4«02 uYsclcctcd male basic airmen 

4.03 unsclcctcd female bnsic ainncu 

4.04 unselected White basic ni.rmcn 

4.05 unsclcctcd non-White basic airmen 



, Identification of Radio Operator Test (ROT) 



4.00 - ROT typ^items 
4.GH - ROT Form Or^c 
4.02^>-l^T Form Twq 




/ 



4.5 



'5.0' 
5.1 

5.2 



NOTL-: 



Idcntirichtioii of AFOQT Samples . ^ , ' 

5.01 -unsclcctcd OTS students . 

5.02 - unselected ROTC students ' • 

5.03 - unselected Category ! and 11 male basic airmen 

5.04 — unselected Category I and 11 female basic airinen 

5.05 - unselected Category I and II White basic airnien ' ' - / 

5.06 -^uiiselected Category I and 11 non-White basic airmen / 

Operatioiiy^f Item Banking System'on UNIVAC 1108 / 

Item banking is maintained by the AFHRL/PHS DS'^ interactive computer user group. All tiles are 
maintairied as public, but they do require the qualifier DS''\ k% ' 

Processor call for item bankings 

"('^DS*Pl:S.lBAr^K FILENAMH. (FILFNAMH supplied by. user) 
.Machine Response . / 

ENTFR TEST 13 NUMBER 
Refer to paragraphs 3.2 jflirough 

Test ID 99.99 may be 'used for learning or demonstration purposes and will not catise update of 
itenl files. 

i ■ ^ , 

Machine responds to test ID witli s ^ ' : 

ITEM POOL CONTAINS NNN ITEMS , 

If item pool is empty or in error, operator is advised, 

TYPE 1 FOR A P-VALUE SEARCH 

TYPE 2 FOR A EATF'nT^TRAIT B-VAEUE .SEARCH, ENTER C^^EQF TO EXJT 
PROGRAM. . ^ 

Values other than i or 2 will cause error message to be printed and the^searcli type question to be 
repeated. v**^ " - t ^ 

If a restart file exists from a previous run, the terminal rcsponj^ 

RFSTART FlEi: AVAIEABEi:, , ^ 

DO YOU WAN T 10 RI-'STARTCY or N)? 

A Y or N for Yes or No response is acceptable, A restart permits an unfniished previcMis run to be 
completed. • ' ^ * 



^^^1 li n e|| ^ e s p o n se 

, ENl ER NUMBER 01' I TEMS REQUIIS'I F:1), MAY NO T I'XCIvI'D 200. ' 

Numi[)ers less than 1 or greater than 200 will cause (juestion to be repealled. 

Mac/fin^: ^Res^ponse ^ ' - 

YOU IIAVP; Si-EIXTITD nnn rfllMS J- ROM NNN RliQUESTIT). I'NTI'R WANTF.D P 
VALUE(or EN PER WAN TI'l) B VALUli). ' 



/ 

^ / 
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Input is checked for range. P-vahic range is .01 to ,99;, B-value range is —3.0 to ■+3.^0. Machine 
responds tq incorrect P value or B value by soliciting value again. 

Sample response to P-valuc search ofP .11. ^ 

OCCIDENTaI MEANS ^ . 

1. UNEXPECTED , '^^^ ' 

2. UNUSUAU ' ' ' . ' • 

3. SUITABLE ■ • 

4. WESTERN ' ' 

5. PERTAINJNG TO life: 

K - .4 P= ,1 1 B = 2,20 D = .30 RB1.85 A -J .64 RPB - .75 C'- .12 SMPL- 1 .00 TEST - 1.00 

DO YOU WANT THIS ITEM? 

May be answered YES or NO or EOF 

A rcsp.onse of EOF will cause a file to be created for later restart. This restart file is catalogued 
by user identification number. 

If YES, machine responds by soliciting next P or B value. If NO, item hank is searched for another 
possible item. After the second item is selected, estimatTons of mean score, variance, standard 
deviation, and reliability arc^computcd and presented. Tiic following message is also presented^: 

PRESS TRANSM?T TO CONTINUE- ' 

If the search procedure is unsvicecssful atjjinding an hc'ni witlnn a.'prescribed interval about the 
reciuested V ox \\ value, a request for a new value will be issued. \ 

After ail the items, have been found and^ accepted by tjie user, the following 'message is issueci'by 
the terminal,- ' ^ . * * ^ " . ' 

ITE^^iS ACCEPTED ARE.: ^ ' / 
^riiis message is followed by a list of the item numbers. The ibll,6wing question is tlicn isi^^tifid, 

DO,YOU WAN r A HARD COPY^OF THESE: 11 EMS? V V 
May be answeled Yl-Sor NO. 

1\ response of NO causes the system to ,be exited with no diangc to item tile, ilie following 
message is then printed on-tiie terniinah ' 

^ ^^= '.MTEM BANKING EXITl- 1) 
NX:)RMALENp V2 
A response V)f Yl'.S is follo\yecl by 

entf:r siiLir riON on ion tq be used eor printing the i iar-d copy 

LISTING. ' ' ' ' 

1 - llstinc; as s[:le,cte:d ~ % 

2 - sort on p valui:s 

3 - sor t ON B VALUERS 

After the oi)tion is selected, the' item llle is updated to show nmnber of the test created for those 
items selected. 'Ilie items are then sent to the high speed printer and the IbJIowing message is. 
printed on th*e ter[ninaL 



■ ■ X . ■ ^ 

***ITEM BANKING EXITED***^ . • . ^ ^ 

N0RMAL£NDV2 / ' - ' % I 

This statement completes the interactive construction of a test. 
5.3 Processor Call for Editor. * 

' ^^^DS*PES.EDIT FILENAME. (FILENAME supplied by user) 
Machine response, 

^ EDIT COMMAND LIST: 

A APPEND TO THE ITEM BANK • / 

Blank GO TO NEXT ITEM 

Exit TERMINATE THE RUN 

F < N > or F all [< N >] . . EDIT ONE OR ALL ^T'' VALyES 

G < N >. GO TO itEM *'N" ^ ^ 1 

H . . /^HELF' (DISPLAY THIS LIST) 

PRESS TRANSMIT FOR MORE INFO 

Machine response 

COMMAND LIST: /' 

K < ANS > r. ..... .CHANGE ITEM KEY VALUE 

N. GO TO NEXT ITEM / 

P< N>R PRINT ITEM "N" 

R< N> OR R. ; ..RETYPE ITEM ^^N'V ' 
T<N>ORT ALL . .EDIT ONEOR ALLTEXTLIN^S . ^ J 

Y ONE LINE HELP . / • 

^ / ■ 

. PRESS TRANSMIT TO RETURNJO EDIT MODE. \ 

Machine response 

ITEM POOL CONTAINS NNN ITEMS 

• \ 
ITEM POOL IS EMl^Y , ; 

then j 

(nmi) EDIT COMMAND? | 

llic nnn is the item number. Tlic operator then gives tlie appropriate edit eommand. 

I 

To enter an item into the pool use the ''Append'' eonnnand and answer tlie maehine prompts. 
Note that only tiie text may be alpharmgrie; all other input fnust be numerie. 

I'o change a line of text in the item on the screen use "Tn" y/hen n is tlic line number. 

To change \\\\ V value use "Pn" when n is the sequential number of the F value. These are the item 
^ statistics 'and item parameters stored with the item. See paragraph 2.0. 

To change the/ keyed item option use tiie ''K" eommand. ''K^r' will change the keyed item option 
U)4. / 

lo ehange tjie entire (^lucstion use the "R" retype option anc\ '-answer terminal prompts as in the 



1? 



\ 



f. ' \ 



Tlic ^JV/' command cfr^nc line help causes the following to be printed. 

, ENTER A, BLANK, E, F, G, H, K, N, P, R, T, 6r Y. . ' 

To tc^iinatc the file editing or building use tlie "EXIT" command. Tlie following will be issued if 
• the JfttAor'is successfully exited. _ 

END ITEME. ITEM POOL FILE CONTAINS nnn ITEMS. 

5.4 Processor Call for Listing Portion qf or Entire Item Bank File. 

/: ' (^^DS*PES,ITEML FILENAME. (FILENAME supplied by user) ' 

Will list' entir^n tern bank file. " 

(^^DS*PES.ITEML FILENAME, A-D, J~M, R-X, Z 
Will list items A-D followed bj-^il-M, R-X, and finally Item Z. 
At the end of this- routiiie, the program will issue the following ipessage: 

END ITEML 

.This program may be used iutcrnctively , as part of a batch ru'n or with the Univac breakpciinting 
procedures. 




^^^^ ::J^^sT¥^' 
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